ABSTRACT
BACKGROUND: Student health centers provide necessary care to university students, yet there is limited literature on efforts to standardize or improve quality of this care. When our urban student health center rapidly shifted to telemedicine-based care at the onset of COVID-19, we designed and deployed two unannounced standardized patient (USP) cases as part of a quality improvement project to assess, broadly, the impact of a new telemedicine workflow and common clinical activities. METHODS: In April and May of 2021, 12 primary care and 4 women's health clinicians conducted virtual visits with two distinct USP cases (N=32 visits, 2 per clinician). Cases included (1) a 21-year-old female requesting a birth control prescription with a positive PHQ-9 and (2) a 21-year-old gaymale who has questions regarding safe sex and vapes regularly. Clinicians were evaluated using two assessments: (1) a behaviorally anchored checklist completed by the USP covering the microsystem and the clinician's skills and (2) a systematic chart review of the electronic health record. RESULTS: Overall, 88% of USPs reported that they would recommend the clinic. Regarding the clinic workflow, 12 of 32 USPs reported receiving a previsit call (clinical protocol), 10 answered the calls and of those, 20% verified two patient identifiers. Between cases, there were significant differences in three domains (mean % well done). Information gathering was significantly higher for the depression case (84% vs. 48%;p=0.004), patient activation was higher for the sexual health case (53% vs 23%;p=0.027), and telemedicine skills were higher in the depression case (77% vs. 57%;p=0.015). Clinician performance was relatively low in case-specific education (depression: 28%, sexual health: 20%) and moderate in management/treatment plan (mean across cases: 66%), relationship development (64%), and education and counseling (65%). In a number of “gold standards of care” items identified by clinical leadership, less than half of clinicians met the target. In the depression case, 56% documented both PHQ-2 and PHQ-9 screenings, 0% screened for alcohol use and quantity, and 66% screened for relationship violence. In the sexual health case, 56% screened for vaping quantity and frequency. 63% counseled on PrEP. No providers completed any part of a physical exam besides commenting on appearance. CONCLUSIONS: Through USP visits and chart reviews, we identified variation in quality of communication skills and use of common screening protocols at this student health center during virtual visits. Health system standards previously integrated into in-person visits may have been lost in the transition to telemedicine care. Our results suggest opportunities exist for targeted improvement efforts that ensure quality virtual care for all students.
ABSTRACT
SETTING AND PARTICIPANTS: Clinician trainees across our health system, including: 1) 107 internal medicine faculty and residents who participated in workplace-based learning at public, private, and federal (Veterans Affairs) ambulatory practices, 2) 16 clinicians at our student health center, and 3) upwards of 250 medical students, residents, and newly-hired general internal medicine (GIM) faculty members from medicine, neurology, and pediatrics departments in our simulation center. DESCRIPTION: While core communication skills have always been at the forefront of medical trainee assessment, information on transference of those skills and integration of the in-person clinical workflow to the virtual care environment was limited prior to COVID-19. NYU Grossman School of Medicine (NYUSOM) implemented a telehealth improvement program across medical students, residents and faculty. In order to assess and improve our systems' ongoing telehealth practices, we employed three distinct educational methodologies across our health systems since March 2020: objective structured clinical exams (OSCEs) and announced (ASPs) and unannounced standardized patient (USPs). Cases were designed to target common, site-specific issues (i.e., hearing loss, COVID-19 vaccine hesitancy, social determinants of health, and sexual and mental health concerns). In line with previous work, all SPs were trained to use a standard behaviorally-anchored checklist to assess communication and telemedicine-specific skills over video visit ( Zoom or WebEx). USPs, professional actors who conduct visits unbeknownst to the clinician, were also trained to collect data on clinic functioning. EVALUATION: Summary reports on performance were provided to both clinical and education leadership and learners to identify future training needs. Data on telemedicine skills across all projects demonstrates room for improvement (mean % marked 'well done' across learners: 46% in the OSCE, 68% at the SHC, and 48% in the public clinics, respectively). Common telemedicine challenges included prompting the SP to adjust their video frame or remove distracting background noise. Most health systems conducted fewer screenings virtually than they did inperson (e.g., at the SHC only 41% and 6% of SPs were screened for alcohol and vaping, respectively;at the public clinics, 25% and 20% were screened for depression and vaping, respectively). Participant feedback reports highlight performance across core domains and provide resources for improvement. DISCUSSION / REFLECTION / LESSONS LEARNED: Our ongoing telemedicine training program demonstrates a highly scalable educational assessment methodology that can be leveraged to optimize common care practices. Data confirm that SPs, ASPs, and USPs can be used across the health care system in simulated and real-world scenarios to identify areas for intervention.